Sense and nonsense about significance testing

Throughout most or all of the empirical sciences, significance testing is one of the most widely used statistical procedures. Unfortunately, it is also widely abused. In recent years, an increasing number of authors have expressed concern over such abuse; see, e.g., the book by Ziliak and McCloskey (2008) who focus mainly on economics, and Kruschke (2010) who focuses on cognitive sciences. In a recent paper, Ambaum (2010) adapted some of the discourse to a climate science setting. While it is important that the theory and practice of significance testing comes under critical scrutiny, it sometimes happens that the criticism goes a bit overboard. Häggström (2010) exposes some examples of this in the case of Ziliak and McCloskey (2008). The purpose of the present note is to evaluate Ambaum (2010) in this respect.